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ABSTRACT 

While educators and policy makers have an intuitive understanding of the influence of 
socioeconomic factors and race on student achievement, these factors make the current emphasis 
on standardized test scores as a primary criterion for evaluating schools and teachers indefensible 
and ineffective. The research presented illustrates the limits of obtaining test score change and 
the impact of socioeconomic status and race on standardized achievement measures. Intentional 
efforts at generating a “success” mindset in students by improving school culture and climate and 
students’ social-emotional and character development are viable steps to be taken alongside 
reductions of socioeconomic inequities. Indeed, these directions become even more important, 
given the long timeframe that such reductions are likely to require. 

Introduction 

Why is it so difficult to create sustained turnaround in troubled schools? Despite the best 
efforts of administrators, teachers, and staff, troubled schools often remain troubled schools. 
Most typically defined as having persistent histories of academic failure (usually a minimum of 
two consecutive years), troubled schools also tend to be schools characterized by relatively high 
rates of violence, relatively poor attendance, and high dropout; they are most typically found in 
high-poverty areas (Education Week, 2014; Hurlburt, Therriault, & Le Floch, 2012). Although a 
difficult task, trying to understand the resistance of these schools to intervention efforts is 
necessary in order to better assist these schools in both making and sustaining changes. Our 
recently completed research project looking at four hundred and eighty three schools across the 
state of New Jersey may provide some insights into this problem. We found a number of real 
world challenges embedded within any efforts at improving test scores. However, inherent in 
these issues are also potential solutions. 

We have watched and assisted as dedicated school administrators and teachers have 
devoted extraordinary time and resources to improve students’ academic perfonnance on 
standardized tests. In preparing students for college and career readiness, school administrators 
must confront two all too familiar achievement gaps. The first is the differential performance of 
students of lower socioeconomic status (SES) relative to their peers of higher SES; the second is 
that of Black and Latino students relative to their White peers. While these challenges may be 
well known, the implications for school administrators of low-perfonning districts and for 
educational policy, have not been sufficiently explicated. Proceeding with conventional wisdom 
is highly likely to lead to results that are ineffective or, at best, short-lived. 

The statistics involved are well known. In the United States, an estimated 22% of all 
children live in poverty with ethnic minorities disproportionately represented. African American 
children represent 14% of all children in the U.S., but constitute 26% of children living in 
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poverty, and likewise, while children of Hispanic origin represent 23% of all children, they make 
up about 32% of children living below the poverty line (U.S. Census Bureau, 2011). One of the 
purposes of the 2001 No Child Left Behind (NCLB) Act was to address the academic 
underperformance of youth who are from lower SES backgrounds and/or are ethnic minorities 
relative to their middle-class, non-ethnic minority counterparts ( NCLB, 2002). NCLB and other 
such initiatives measure success by relying on standardized test scores. 

However, there is legitimate concern that the relationship between SES, ethnicity, and 
achievement scores presents a serious dilemma when attempting to improve these scores. How 
do you improve the test scores of disadvantaged groups when it is their disadvantage that is 
influencing their performance on standardized tests? Educators are faced with fighting against a 
larger system of racial and economic issues that are rooted in a deep-seated history. But more 
than that, as the present study illustrates, there are statistical challenges that await well- 
intentioned efforts at turning around troubled schools. These challenges make it difficult to 
accurately measure intervention success using the typical array of standardized test scores. 

In studies that have explored this gap from a wider perspective, the defining feature is 
typically the socioeconomic resources of those communities (Rothstein, 2004; Rumberger & 
Palardy, 2005). Socioeconomic status and ethnicity frequently are found to be interrelated, as 
students of Hispanic and Black ethnicity are often found in segregated, high-poverty schools 
with limited resources (Orfield & Lee, 2004). 

The Current Study 

School-level factors such as classroom size (Ehrenberg, Brewer, Gamoran, & Willms, 
2001), school size (Leithwood & Jantzi, 2009) and teacher mobility (Borman & Dowling, 2008) 
have been shown to influence academic outcomes. However, while these factors are frequent 
targets of policy aimed at improving academics, the impact of a school’s socioeconomic status 
and racial make-up on standardized test scores represent a metric by which to measure the 
success of such policies. If the NCLB Act has had success, then a school’s percent of students 
passing a standard achievement test should be impacted more by these mutable factors rather 
than the socio-economic and racial make-up of the school - or at least we would hope to see an 
improvement over the 1966 data from the Coleman Report (Coleman et ah, 1966). The purpose 
of the current study is to detail the independent influence of socioeconomic status and race, have 
on achievement scores. The results indicated that SES and race continue to impact school level 
test scores above the other predictive school level factors of class size, school size, and teacher 
mobility. 


Methods 


Setting and Participants 

The data presented here utilizes the school as the unit of analysis, thus all factors 
examined are at the school level rather than the individual, mimicking NCLB’s practice of 
evaluating schools. The data is from the 2009-2010 academic school year for 144 schools which 
included an 8th grade but not a 3 rd grade or 9 th grade (referred to as middle school). The majority 
of schools reflected grades 6-8 (see Table 1 for school demographic factors). 
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Table 1 

School Demographic Factors 


Middle School Sample (n = 144) 



M 

SD 

Range 

Total Enrollment 

701.08 

290.83 

203 - 

■ 1879 

Average Class Size 

19.52 

3.72 

6.30- 

-27.40 

Faculty Mobility 

5.04 

10.57 

0.00- 

109.40 

% Free or Reduced Lunch 

24.40 

23.04 

0.00- 

- 86.42 

% Female 

48.49 

2.40 

41.67 

-55.46 

% White 

60.12 

27.73 

0.11 - 

-95.86 

% Black 

14.58 

20.06 

0.00- 

-95.18 

% Hispanic 

15.55 

17.59 

0.00- 

-95.15 

% Asian 

8.73 

9.51 

0.00- 

-44.96 

% Proficient and Advanced Proficient on Language 

85.34 

13.12 

23.90- 

- 100.00 

% Proficient and Advanced Proficient on Math 

71.77 

16.04 

13.10 

- 97.60 


Measures 

All variables came from two publically available data sources: 1) The New Jersey 
Department of Education (NJDOE) School Report Cards online database (State of New Jersey 
Department of Education, 2010) or 2) the Institute of Education Sciences (IES) of the National 
Center for Education Statistics (NCES) public school online database (Institute of Education 
Sciences, 2010). All predictor variables were centered by grade level to reduce multicollinearity. 
NJDOE variables included and analyzed for this study were total enrollment of students, average 
class size, faculty mobility rate, and standardized achievement tests. NCES variables included 
and analyzed for this study were free and reduced lunch status, race/ethnicity proportions, and 
male/female proportions for each school. 

Public school students in New Jersey take the NJ Assessment of Skills and Knowledge 
(NJASK) each year and the current study reflects data from 8 th graders tested in April-May, 
2010. The NAEP, described as a “gold standard” for monitoring the educational progress of 
American students (Jones, Olkin, & American Educational Research Association, 2004), and the 
New Jersey state assessments differ in how some of the content and skills are measured as well 
as the method used for setting performance standards (i.e., the cut points for detennining 
achievement levels). However, it is generally acceptable to consider the New Jersey state rating 
of “proficient” as comparable to a NAEP “basic” rating (U.S. Department of Education, Institute 
of Education Sciences, & National Center for Education Statistics, 2010). Finally, while these 
tests yield scaled scores for each individual taking them, the data presented here reflect school- 
level percentage of students who were proficient or advanced proficient. The criterion of 
“proficient” was used because this qualitative label is used to detennine accolades and sanctions 
by the public and the government. 
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Data Analysis 

Schools used in this study were part of the Developing Safe and Civil Schools Project 
(DSACS) or were identified as matched controls for a DSACS school, based on demographic 
variables including district, size, grade configuration, and other factors. The DSACS project was 
a publically funded, voluntary initiative aimed at improving school climate. The current study 
utilized the existing database developed for this project, but does not assess the DSACS project 
itself. All data were downloaded from the publically-available New Jersey Department of 
Education (NJDOE) School Report Cards online database (State of New Jersey Department of 
Education, 2010) and Institute of Education Sciences (IES) of the National Center for Education 
Statistics (NCES) public school online database (Institute of Education Sciences, 2010) for the 
school year 2009-2010. Schools were included in the study if they met grade-level inclusionary 
criteria and were not missing any variables of interest. Data were analyzed using SPSS version 
20 . 


Results 

Hierarchical Regression Analysis Predicting School Language Proficiency 

To test the hypothesis that the percentage of students in a school who are at the proficient 
or advanced proficient level on the NJ state language test is a function of the percentage of Black 
and Hispanic students after controlling for other school demographic factors, a hierarchical 
regression analysis was performed for each school type. Percent female, faculty mobility, total 
enrollment, and average class size were entered first, followed by the percentage of students 
receiving free or reduced lunch, and then the percentage of students who were Black and the 
percentage of students of Hispanic origin, independently; in the last step, interaction terms for 
percentage of students receiving free or reduced lunch by each of the two racial/ethnic groups 
were entered to help determine whether the relationship between race and test scores is 
moderated by SES. All continuous predictor variables were centered to reduce multicollinearity 
for hierarchical regression. All percent variables were coded on a 0.00-100.00% scale, so a one 
unit change on any of the predictor variables reflects a one unit change in the percentage of 
students in a school who are at the proficient or advanced proficient level on the language test. 

For this study, we present data from middle schools, as these schools represented the 
modal trend shown in the data across school types. As shown in Table 2, the school demographic 
factors in Step 1 accounted for 17% of the variance in language proficiency. SES added an 
additional 58% of variance explained in Step 2 (R 2 change = 0.58, F = 319.60, p < .001). The 
addition of percent Black and Hispanic students in Step 3 increased the variance explained from 
75% to 81% (F = 20.90, p < .001). Furthermore, the addition of the interaction between race and 
SES increased the variance explained by an additional 4% (F= 16.95,/? < .001). 

In the final model, the faculty mobility rate (b = -.12, p = 0.008), the percentage of 
students receiving free or reduced lunch (b = -0.16, p = .001), the percentage of students of 
Hispanic origin (b = -0.26, p = .001), and the percentage of Black students (b = -.13, p = .003) 
were all significant predictors of language proficiency, holding all other variables constant. 
Specifically looking at the impact of race, with every 10% increase in the percentage of students 
of Hispanic or Black ethnicity in a school, on average there is a 2.60% or 1.30% decrease, 
respectively, in the percentage of students who are proficient on the language test. In addition, 
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the interaction between the percentage of Black students and free/reduced lunch status was 
significant (b =-0.01 ,P< .001), suggesting that the relationship between the percentage of Black 
students and language test proficiency is significantly and negatively moderated by the 
percentage of students receiving free or reduced lunch. The higher the percentage of students 
receiving free or reduced lunch in a school, the more negative the relationship (or slope) between 
the percentage of Black students and language proficiency (Figure 1). The interaction between 
the percentage of Hispanic students by free/reduced lunch status was not significant. 


Table 2 


Summary of Hierarchical Regression Analysis Predicting School Language Proficiency (% of school at the proficient or advanced 
proficient level) for Middle Schools (n = 144) _ 




Model 1 



Model 2 



Model 3 



Model 4 


B 

SE 

B 

B 

B 

SE 

B 

P 

B 

SE 

B 

P 

B 

SE R 

B P 

Constant 

84.79*** 1.02 


85.61 

*** .56 


85.62 

*** .50 


87.69*** .65 

% Female 

.43 

.43 

.08 

.005 

.24 

.001 

.OS 

.21 

.01 

.03 

.19 .005 

Faculty Mobility 

-.21 

.10 

-.17* 

-.19 

.05 

-.15** 

-.18 

.05 

.14*** 

-.12 

.04 -.10** 

Total Enrollment 

.002 

.004 

.04 

-.001 

.002 

-.03 

.00 

.002 

.004 

-.002 

.002 -.04 

Average Class 

1.23 

.28 

.35*** 

.07 

.17 

.02 

.11 

.15 

.03 

.14 

.14 .041 

% Free or 
Reduced Lunch 




-.48 

.03 

.84*** 

-.21 

.05 

.37*** 

-.16 

.05 -.28** 

« b Black 







-.26 

.04 

.39*** 

-.13 

.04 -JO** 

% Hispanic 







-.23 

.05 

.30*** 

-.26 

•° 7 J5*** 

« o Black x % 

Free/Reduced 
Lunch 










-.006 

.ool 

% Hispanic x % 
Free/ Reduced 
Lunch 










-.001 

.001 -.06 

R 2 


0.17 



0.75 



0.81 



0.85 

Change in R 2 


0.17 



0.58 



0.06 



0.04 

F for R 2 change 


7.21*** 



319.60*** 



20.90*** 



16.95*** 


Note. All predictor variables are centered; “p < .05, x *p < .01, ***/> < .001 


It is important to note that race/ethnicity, without SES interaction, did not become 
significant until middle school. And while elementary school test scores were significantly 
affected by percent free or reduced lunch, this effect was more significant in language than math. 
However, by high school, the impact of the racial make-up of the school became highly 
significant, particularly in math achievement. Additionally, for middle schools and high schools, 
SES was only a significant moderator between race/ethnicity and test proficiency when looking 
at the percentage of Black students in school, and generally not with students of Hispanic 
origins. 



SURMOUNTING THE CHALLENGES 


19 


u 

C 

o 


0 

c 

n 

■a 


.a 

u 


O. 



Low % Free/Reduced 
Lunch 

High % Free/ 
Reduced Lunch 


Note. All predictor variables were centered. 


Figure 1. Percentage of Black Students and Percentage of Students Receiving Free and Reduced 
Lunch as Predictors of the Percentage of Students at the Proficient or Advanced Proficient Level 
on the Standardized Language Test in Middle Schools (n = 144) 


Discussion 


Consistent with prior research (Campbell, Hambo, & Mazzeo, 1999; Campbell, Pungello, 
Ramey, Miller, & Burchinal, 2001;), the current study found that race/ethnicity accounts for a 
significant and meaningful amount of variance in students’ test scores. This significance was 
over and above a highly significant amount accounted for by school demographic factors and 
SES. Additionally, results indicated that SES moderated the relationship between race and test 
scores, and that the interaction between race and SES gains significance by 8 th grade, 
representing a medium effect size increment in language proficiency test scores. Our data also 
suggest that the interaction effect strengthens in high school. Clearly, students in schools with 
greater Black and Latino populations experience an especially challenging educational climate. 
As well, ethnicity and the interaction between it and SES significantly explained additional 
variability in test scores for the high schools in our sample, suggesting that something else is 
going on beyond the impact of SES on scores. 

Specifically, we found that the effect of ethnic make-up on school test scores was 
stronger in schools with higher rates of students receiving free or reduced lunch. This, in essence, 
indicates that high poverty, high minority population schools face a particular constellation of 
factors that when combined, make showing progress in substantial academic test score 
perfonnance extremely difficult. Given this, questions can clearly be raised in education policy 
around the appropriateness of the emphasis on high-stakes testing in the United States, and 
particularly doing so on the kinds of assessments that are currently administered. Further, the 
way in which test scores are used in evaluating teachers from schools in low SES areas with a 
relatively high percentage of students of Black and Hispanic ethnicity requires considerable 
rethinking. 
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Reversing the Inequities for which Test Scores are a Proxy 

From the data, but equally so from our reading of the literature and our experience 
working in diverse schools over many years, we suspect that certain mechanisms are likely to be 
operating at the intersection of ethnicity and SES. It is these mechanisms that can and must be 
effectively targeted for change. At the individual level, recent work has identified the pervasive 
impact of racial microaggressions, the “subtle insults (verbal, nonverbal, and/or visual) directed 
toward racial minorities, often automatically or unconsciously” (Solorzano, Ceja, & Yosso, 2000) 
that are hidden in everyday interactions and widen the gap of racial realities. The cumulative 
nature of these innocuous expressions is detrimental to racial minorities as, inherent in the 
ambiguous nature of the aggressions, the interaction causes stress which then impairs 
perfonnance in multitude of settings (Omi, 1994; Sue et al., 2007). Indeed, studies have found 
that interpersonal racial oppression has been found to hann the mental well-being and academic 
performance of minority students (Rovai, Gallien Jr, & Wighting, 2005). 

These mechanisms are relevant not just at the individual level but within the context of 
the school. While the current study explored only the school level impact of SES, a large scale 
study of Austrian students found that as the mean SES of a school increased, there were 
consistent increases in students’ academic achievement, and that this relationship was similar for 
all students regardless of their individual SES (Perry & McConney, 2010). Further, in the larger 
context of schools, there is the internalized oppression that results when children are in an 
environment in which they feel devalued and inferior and perceive little or no likelihood of their 
status changing (Garcia Coll et al., 1996; Kloos et al., 2012). Under such circumstances, children 
are likely to experience a mindset of academic defeat, rather than the tenacity or perseverance 
that has been empirically linked to their success (Blackwell, Trzesniewski, & Dweck, 2007; 
Farrington et al., 2012). What appears to happen, as reported by Ou and Reynolds (2008) in the 
Chicago schools, is that students engage in a vicious cycle of lowering academic aspirations, 
which eventually, if indirectly, leads to lower employment expectations, lowered health and life 
aspirations, and the informal enforcement of anti-achievement norms and stereotype threat that 
serves as a self-fulfilling prophecy for those who attempt to break out of the pattern. 
Alternatively, those students who have not internalized oppression are more likely to transfer to 
another—hopefully more positive—school environment, in the unusual case that their external 
circumstances allow. 

Additionally, perceptions of the culture and climate within the school may provide a 
partial explanation for the achievement and discipline gaps across ethnicities. This is because 
ethnic minority students have been found to perceive their environment as less safe and report 
lower levels of achievement motivation than White youths, even after controlling for classroom- 
and school-level factors (Koth, Bradshaw, & Leaf, 2008). It is not a reasonable expectation for 
ethnic minority youth who do not feel safe or valued in school to sustain a strong commitment to 
learning, particularly when also aware of the long-standing gap between themselves and their 
White, advantaged peers. Furthermore, if the adults, whose professional role is to educate them, 
accept them through open school doors for 180 days each school year but cannot provide a 
welcoming and supportive environment, or at least keep them safe, what can these students 
reasonably expect from the wider society? 

Research is showing with increasing clarity that the school environments within which 
students learn exercise great influence on them (Thapa, Cohen, Guffey, & Higgins- 
D’Alessandro, 2013). Children ultimately are hopeful; indeed, the elementary school-level data 
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from our study do not show the same level of achievement gap, and there is reason to believe 
that a supportive school environment within which students can learn core social, emotional, and 
academic skills may be a catalyst for a true turnaround process (Cohen & Elias, 2011; Zins, 
Weissberg, Wang, & Walberg, 2004). 

Social and emotional learning (SEL) can be defined as the capacity to recognize and 
manage emotions, solve problems effectively, set and achieve positive goals, appreciate the 
perspectives of others, establish and maintain positive relationships, make responsible decisions, 
and handle interpersonal situations constructively (Elias et ah, 1997). It follows from this 
definition that social and emotional competencies are a combination of behaviors, cognitions, 
and emotions that can be seen in the effective application of the knowledge, attitudes, and skills 
necessary to recognize and manage emotions; have and express care and concern for others; 
ability to make responsible decisions; establish positive relationships; and ability to capably 
handle challenging situations (Zins & Elias, 2006). 

It is difficult to imagine a classroom, or any school context, that can be engaging and 
productive in the absence of students’ possession of these competencies. But these capacities 
cannot be learned informally or haphazardly. They must be learned systematically and in schools 
with climates and cultures that value student competence in areas other than academic content 
(Cohen & Elias, 2011). These skills are learned best and most deeply when students are in 
collaboration with their teachers and learning cooperatively with their peers. Schools attempting 
a turnaround, or otherwise seeking to better their students’ academic performance, must realize 
that academic development cannot be fostered unless students’ social-emotional and character 
development is also fostered (Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, 2011). This is 
especially true of schools attempting to implement the Common Core in contexts with poor 
histories of academic success (Elias, 2014). 

Recommendation for Policy and Practice 

To have a chance of countering the powerful convergence of SES and ethnicity on 
disadvantaged learners, school administrators must support evidence-based and continuous 
interventions. These interventions can and should work to improve the culture and climate of 
schools and address students’ sense of meaning and purpose, voice and value, and the social, 
emotional, and character competencies needed to enact the opportunities students are given. 
Students must be actively involved in shaping policy. This is true for the range of school-based 
interventions, from service learning, to cooperative learning, to student involvement in designing 
classroom and school rules addressing school problems such as harassment, intimidation and 
bullying, drugs and alcohol, obesity, dropout, or lack of academic support at home. This 
involvement builds a sense of civic participation that can be a source of empowerment—a sense 
of “I can”—upon which academic skills may be built. 

This is not a small point; if students do not believe they can succeed or that it benefits 
them to succeed, it will not matter how much the curriculum is revised and assessments are 
changed (Farrington et ah, 2012). There must be “both/and” thinking in our educational policy 
and practice. The urgency of improving students’ academic skills does not automatically mean 
educators can or should focus all of their efforts on those skills. Indeed, that is a prescription for 
failure because we lead our youth into 180 days each school year of immersion in their 
deficiencies. How can this be a source of positive motivation and turnaround? 

Our findings are sobering, and do not absolve those in power and those who make policy from 
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reducing the socioeconomic inequalities in our society, creating more and more visible pathways 
to success for our most disadvantaged youth, and rethinking an inherently unfair testing regimen. 
But for educators now, particularly those in leadership roles, our findings also point to a set of 
large and deeply entrenched obstacles to students’ academic success. These obstacles cannot be 
blasted through with the usual tools of academic remediation. They must be eroded persistently 
through a nurturing school culture and climate, and then bypassed by providing a solid emphasis 
on social-emotional learning and character development that will provide students with the 
fortitude and grit to face and surmount the tests of life. 
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